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the criterion setting (66%) of the Prescriptive Reading Inventory 
(PRI), a criterion referenced reading test., The subjects, who were 
involved in a larger experimental study that used the PRI asa 
criterion measure, were 92 third and fifth grade students. Initial 
assessment with the Comprehensive Tests of Basic Skills revealed 
varying student achievement that necessitated the administration of 
various levels of the PRI on a pre/posttest basis. After. these tests, 
the students were classified as masters/nonmasters using three 
séparate criterion’ scores (66%, 75%, and 90%). The Harris Index of 
Efficiency was computed for each criterion sccre to determine which 
score proved most efficient at sorting the pupils into two separate 
groups. The results suggest that the criterion sccre cf 66% was the 
most efficient cutting point with the PRI; a such less efficient 
sorting of the subjects was evident when using the 75% and 90% 
levels. The results also indicate that the Harris Index is an: 
effective descriptive measure that can help to determine at what 
cutting point a test might yield the sost infcrmaticn for 
instructional purposes. (RL) 
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The notion af 4 obiceeton referenced test has kindled considerable 
enthusiasm among those concerned Staats with reading assessment (James, 
1975). Stmabtaaenus attempts, however, to implement those concepts have 
given rise a a good deat of controversy. See, for instance, Dole's (1974) 
response to a recent Thompson and Dziuban (1973) Raper and their rebuttal 
(Thompson and Dziuban, 1974). The futility of trying to igo out" various 
claims cavaniliug the use sid nature of a so-called criterion referenced test 
is becoming more and more obvious. A serious student of the movement might 
be easily overwhelmed by the currently available literature, large portions 
of which are something less than underkeandania, Because of this we shall 
not outline the definitions which abound in the literature, since here we 
are not so much concerned with a criterion test as with a criterion score. 

The problem ‘addressed involves working with an assessment technique 
(test) aiek can be used to sort students into a minimum of bs groups-- 
mastery and scarey Obviously this necessitates the selection of some 
criterion or cutting score. A more realistic situation might involve the 
setting of two scores one which designates mastery when students score above 
it and a separate score below wiih sieatttes non-mastery (Harris, 1974).° 
Those falling between the two scores would be designated neither and for 


them judgment might be suspended. For this paper we shall consider only 


the one criterion score case. The problem is to pick the smallest proportion 


of items passed which might indicate that the pupil has mastered the 


objective(s). 


An Index of Efficiency 
Recently Harris (1974) developed an Index of Efficiency for a fixed 


length mastery test Gusts By the selection of a cutting score on a K 
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item device we are able to sort a sample of pupils into two groups and 
evaluate the efficiency of the decision. Harris (1974) built upon 


Fisher's (1936) two group discriminant function where the column vector 


weights were chosen to be L's, this creating a special cant equivalent to 
incorporating the proportion of items passed as the "criterion" fox dis- 
eriodeating between masters and non-masters. He showed that this was related 
to the ssh canonical correlation, u2, which in this case was the squared 
person product moment correlation between the total test scores and a 
"dummy" variable indicating the mastery or a aeatcies membership of the 
‘pugtie tt The index is positive lying between “0 wal 1 with the upper 
limit being achieved wen only two apie Shed in“s group. Harris (1974) 
showed that I a highly desirable features, First, ” can be con- 
ere as ne of true to observed score varnishes ealewing it to 
classic reliability theory. Secondly, the largest u2 for a particular 
test is the upper limit to validity when such validity is defined in a 


like manner. It is the purpose of this paper to report the results of the 


application of the index to a "criterion referenced reading" test. 


the Test, Subjects, and Methods 

The Prescriptive Reading Inventory (PRI) is published by California 
Test Bureau/McGraw-Hill (1972) under the term criterion referenced test. 
The instrument is organized into four levels which have associated grade 


level equivalents. It is an untimed test and is purported to facilitate 


(1) the reader will recognize this as the squared point-biserial correlation. 
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the diagnosis of reading skill SRE EHERE ERY Mastery for a particular 
objective is based upon a criterion score usually sixty-six per cent of 


the items passed--customarily two out of three questions per objective. 


The subjects were involved in a larger experimental ‘study in which the 


‘. PRI was used as a criterion measure. They were ninety-two students in 


grades three and five attending a central Florida elementary school. Of 

\ , ; 
this number thirty-five were boys and fifty-seven were girls.. Initial 
assessment with the Comprehensive Test of Basic Skills revealed discrepancies 


in the achievement ranges of the groups and necessitated administration of 


various levels of the PRI on a pretest/post test basis: 


Appropriate Grade a 
Level of 
Test Level the Test - Administration Groups 
Red (A) ~o-2., 3rd Grade-One Group 
Green (B) 0-3. 3rd Grade-One Group 


1.5-2.5 

2.0-3.5 
Blue (C) 3.0-4.5 5th Grade-Two Groups 
Orange (E) 4.0-6.5 ‘Sth Grade-Two Groups 

Upon completion of the administration of the pre and post test for each 
subject, a percentage of items correct (P.C.) for the total test was determined: 
PC, EFT 

where P equals the number of items passed and T is the total number of items 
on the test. Based.upon these scores each student was classified as a master 
or non-master using three separate criterion scores: 66%, 75%, 90%. For 
a given student if his P.C. score equaled or exceeded the criterion, he was 
assigned a 1 (master) and a O (non-master) if his score was below the cutoff 


point. For each group the Harris Index of Efficiency was computed for each 


criterion score fot both the pre and post tests. The objective was to 


” 


determine which criterion score proved most efficient for sorting pupils 


a 


into two separate groups. 


Results 


bd 
~ 


The results of the analysis are presented in Table One. 


Insert Table One About Here © 


It may be observed for the third grade red — that the most efficient . 
criterion score. for both the pre and post test was at the 66% level’ (.56 and 
-81). Substantial decrease in the index wane moked as the criterion score was 
raised. The coefficient for the npetwet at the 90% level was ihdwbinea, This 

| simply means that -no student achieved the 90%, beeats therefore, «11 must be 
classified as non-masters, so at this score the test te eomnterely inefficient 
for two aréuis purposes. A similar Kleden may _ | for the third 
grade green group with the’ largest index obtained for both the pre and baile 
test at the 66% criterion score (.68 and .7/6). The fifth grade blue zope 
exhibited a similar pattern in that the 66%: level proved’ to be the most 
"efficient cutting point of the three. Again a aibveneiat drop-off was 
noted as the.criterion score was raised fo’ 752 and 90%. The highest foiltees 
for the first blue group were -60 and 73 ana sont -60 rid «h3 for the second 
blue group. The fifth grade orange groups ceveatea somewhat different results . 
in that all indices for the pretests were undefined as — were for all tests 
at the 90% level. Once south maximum efficiency was achieved at the 667 


cutting score (u2 = .71, .71). Another interesting trend is observable 
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made. 
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in that there appeared to be a substantial increase in the indices from the 


pre to the post tests in all instances where such determination could be 


- 


Discussion 


The results af ‘this study seem to suggest that with this sample’ of 


4 


subjects the most efficient cutting point with the PRI was achieved ata 


eriterion score of 66%. This corresponds to the level Secommended by the 


test publishers. If one chose to use the 75% or 90% levels, a ‘much less ‘ 


ne 
© 


efficient sorting of subjects sion be realized. ‘These scores were deter- 


mined using full scale scores (based upon all dena and were not related 


be.much less efficient in providing information as to which students were 


to specific objectives. It is our speculation. these shorter tests would 


e 


masters and non-masters. 


. It seems clear that the Harris Index. as a dest¢riptive measure can be 
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used effectively to help determine at what cutting point the test might 


yield the most information for instructional purposes. A strategy might 


‘involve ‘for a particular group first determining where the test is most 


efficient and subsequently deciding whether or not that score can be ts 
accepted as mastery level. For instance, pe oie conceivable that a particular 
test might achieve ies maximum efficiency at a score ite 45% but ‘nae 
teachers or curriculum spextatiars are unwilling to accept that as a minimum. 
mastery score. Further consideration would be necessary eee eae use 

of the test and design of the reading program. “Additionally it is very. 


likely that the same criterion score will not be. equally as efficient for 


- 


all ability levels, so that the combination of the difficulty of the test 


and the skill level of students can have a marked effect on the efficiency 
and thus diagnostic power of an instrument. We point out that we have used- 
the term criterion referenced test very sparingly. We have followed 


Harris’ (1974) example, dealt with a criterion score, and demonstrated 


that his Index ‘provides valuable information regarding such scores. 


‘ , 


Accordingly, those concerned with reading mastery and instructional pet 
- decisions related to it might find these procedures of considerabfe ae 
assistance, 
: 


TABLE ONE 


Harris Indices (3) for Each of the Test Administrations* 


= ere a 2 ee 
fon |me [tear] ve | foot] pre | Post | pee | toot 
| ’ 
68 : * 273 


¥ * indicates that the index was undefined 
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